Lidar based communication

ABSTRACT

Systems and methods for performing operations based on LIDAR communications are described. An example device may include one or more processors and a memory coupled to the one or more processors. The memory includes instructions that, when executed by the one or more processors, cause the device to receive data associated with a modulated optical signal emitted by a transmitter of a first LIDAR device and received by a receiver of a second LIDAR device coupled to a vehicle and the device, generate a rendering of an environment of the vehicle based on information from one or more LIDAR devices coupled to the vehicle, and update the rendering based on the received data. Updating the rendering includes updating an object rendering of an object in the environment of the vehicle. The instructions further cause the device to provide the updated rendering for display on a display coupled to the vehicle.

TECHNICAL FIELD

This disclosure relates generally to LIDAR devices, and specifically tousing LIDARs as a communication channel in self-driving applications.

DESCRIPTION OF THE RELATED TECHNOLOGY

A light detection and ranging (LIDAR) device may be used to detectobjects in an environment. For example, a vehicle may include or becoupled to one or more LIDAR devices to detect, e.g., vehicles,pedestrians, traffic signals, obstacles, etc. A LIDAR device emits lightat a specific frequency (such as at 800-1000 nm or at 1550 nm), and theLIDAR device receives reflections of the emitted light. The LIDAR devicethen determines a time-of-flight (ToF) of the light to estimatedistances of a number of reflective surfaces while scanning through anenvironment. The estimated distances may be used to generate a pointcloud representation of the environment or otherwise be used to renderthe environment or assist with vehicle operation.

For example, vehicles can be configured to operate in an autonomous modein which the vehicle navigates through an environment with little or noinput from a driver. An autonomous vehicle may include one or more LIDARdevices to determine distances of objects in the environment, and thedistances may be used for controlling navigation of the vehicle. Inanother example, a vehicle may include one or more LIDAR devices toassist a driver, such as in performing adaptive cruise control,providing cross-traffic warnings, providing lane departure warnings,etc. up to performing all safety-critical driving functions andmonitoring roadway conditions in a fully-autonomous system.

SUMMARY

This Summary is provided to introduce in a simplified form a selectionof concepts that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tolimit the scope of the claimed subject matter.

Innovative aspects of the subject matter described in this disclosurecan be implemented for a device coupled to one or more LIDAR devices. Insome implementations, an example device includes one or more processorsand a memory coupled to the one or more processors. The memory includesinstructions that, when executed by the one or more processors, causethe device to receive data associated with a modulated optical signalemitted by a transmitter of a first LIDAR device and received by areceiver of a second LIDAR device coupled to a vehicle and the device,generate a rendering of an environment of the vehicle based oninformation from one or more LIDAR devices coupled to the vehicle, andupdate the rendering based on the received data. Updating the renderingincludes updating an object rendering of an object in the environment ofthe vehicle. The instructions further cause the device to provide theupdated rendering for display on a display coupled to the vehicle.

Innovative aspects of the subject matter described in this disclosurecan be implemented in a computer readable medium. The computer readablemedium stores instructions that, when executed by one or more processorsof a device, cause the device to receive data associated with amodulated optical signal emitted by a transmitter of a first LIDARdevice and received by a receiver of a second LIDAR device coupled to avehicle and the device, generate a rendering of an environment of thevehicle based on information from one or more LIDAR devices coupled tothe vehicle, and update the rendering based on the received data.Updating the rendering includes updating an object rendering of anobject in the environment of the vehicle. The instructions further causethe device to provide the updated rendering for display on a displaycoupled to the vehicle.

Innovative aspects of the subject matter described in this disclosurecan be implemented as a method. An example method includes receiving, bya device, data associated with a modulated optical signal emitted by atransmitter of a first LIDAR device and received by a receiver of asecond LIDAR device coupled to a vehicle and the device, generating arendering of an environment of the vehicle based on information from oneor more LIDAR devices coupled to the vehicle, and updating the renderingbased on the received data. Updating the rendering includes updating anobject rendering of an object in the environment of the vehicle. Themethod further includes providing the updated rendering for display on adisplay coupled to the vehicle.

Another example device may include one or more processors and a memorycoupled to the one or more processors. The memory includes instructionsthat, when executed by the one or more processors, cause the device toreceive data associated with a modulated optical signal emitted by atransmitter of a first LIDAR device and received by a receiver of asecond LIDAR device coupled to an infrastructure (such as a toll booth,road construction area, tunnel entrance, etc.) and the device. Thedevice may generate a rendering of an environment of the infrastructurebased on information from one or more LIDAR devices coupled to theinfrastructure. The device may also update the rendering based on thereceived data. Updating the rendering may include updating an objectrendering of an object in the environment of the infrastructure. Thedevice may further provide the updated rendering for display. Forexample, the rendering may be displayed for a toll booth attendant, acentral traffic office comptroller, a construction worker foreman at aconstruction site, etc.

The data may include an indication of the object in the environment, andupdating the object rendering may include highlighting the objectrendering during display, adjusting the texture of the object rendering,including a representative image of the object in the rendering of theenvironment, and/or adjusting the dimensions of the object rendering.Displaying the updated rendering may include notifying a viewer that theobject in the environment of the infrastructure is an emergency vehicle.

The device may also determine navigation operations for one or morevehicles in the infrastructure's environment based on the received data,and the device may provide the provide the adjusted navigationoperations to the one or more vehicles. The modulated optical signal maybe received by the second LIDAR device operating in a communicationsmode, and objects in the infrastructure's environment and line of sightof the second LIDAR device may be sensed by the second LIDAR deviceoperating in a detection mode.

The device may further receive a second data associated with a secondmodulated optical signal received by the second LIDAR device, and updateone or more entries in a local database based on the received seconddata. In some implementations, the device may transmit data regardingthe updates to other LIDAR devices for updating a database local to theother LIDAR devices.

Details of one or more implementations of the subject matter describedin this disclosure are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages will becomeapparent from the description, the drawings and the claims. Note thatthe relative dimensions of the following figures may not be drawn toscale.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example LIDAR device.

FIG. 2A illustrates an example LIDAR device in an environment.

FIG. 2B is an example timing diagram of waveforms corresponding toemitted light pulses and received light pulses of an example LIDARdevice.

FIG. 3 illustrates an example packet format for a plurality of receivedpackets from a LIDAR transmission based on an example communicationprotocol.

FIG. 4 illustrates an environment including an example parking space foran autonomous fleet vehicle to return to when not in use.

FIG. 5 illustrates an example environment for communicating the presenceof an ambulance to another vehicle in the environment.

FIG. 6A illustrates an example environment of a vehicle.

FIG. 6B illustrates an example rendering of the environment of thevehicle as sensed by a LIDAR device.

FIG. 6C illustrates an example environment of a vehicle including anemergency vehicle.

FIG. 6D illustrates an example rendering of the environment of thevehicle in FIG. 6C as sensed by a LIDAR device.

FIG. 7 shows a flow chart depicting an example operation for adjusting arendering of an environment of a LIDAR device.

FIG. 8 illustrates an example adjusted rendering of the environment inFIG. 6A.

FIG. 9A illustrates an example rendering of the environment in FIG. 6Awhere the two vehicles in the environment belong to the same fleet.

FIG. 9B illustrates an example rendering of the environment in FIG. 6Cwhere the environment includes an emergency vehicle.

FIG. 10 shows a flow chart depicting an example operation for performingone or more operations based on data received in a transmission from aLIDAR transmitter.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

Efforts have been made to couple vehicles and infrastructure viacellular communications. For example, some vehicles include a cellularmodem to communicate with other vehicles or devices over the 5 GHz radiospectrum. In another example, a vehicle may be equipped with DedicatedShort Range Communication (DSRC) equipment to communicate with othervehicles or devices over the 5.9 GHz radio spectrum. One problem withcellular or radio technology for communication is that the vehicle mayrequire a cellular connection or require another vehicle orinfrastructure to be equipped with consistent technology forcommunicating. For example, some rural environments or saturated urbanenvironments may not include an available cellular connection, and avehicle may be unable to communicate with others via its cellular modem.Another problem with cellular technology for communication is theinherent latency associated with the communications. For example,cellular communications between vehicles may require communicatingthrough one or more base stations of a cellular network, which may delaycommunications. Additionally, the wavelength of radio signals forcommunication limits the communication bandwidth. Furthermore, cellularand radio signals are typically transmitted omnidirectionally, andtransmitting a plurality of omnidirectional signals over the air canquickly saturate the environment.

Many vehicles and infrastructure (such as toll booths, traffic signals,charging stations, etc.) can include or be coupled to one or more LIDARdevices. Additionally, as autonomous vehicles become more commonplace,equipped LIDAR devices will also become more commonplace. In someaspects, a LIDAR device may be configured to communicate with otherLIDAR devices in addition to performing ToF measurements (for detectingsurfaces in the environment). Since emitted optical signals have ahigher frequency than radio signals, communication between LIDAR devicesmay have a higher communication bandwidth than cellular communications.Additionally, communication between LIDAR devices does not require acellular network or other infrastructure. Furthermore, optical signals(such as signals with a wavelength close to 1000 nm) may be emitted witha focused dispersion pattern to prevent saturation and interference frommultiple optical signals being transmitted concurrently over the air.

Implementations of the subject matter described herein may allow a LIDARdevice to communicate with another LIDAR device (called “LIDARcommunications” herein). LIDAR communications may be used forvehicle-to-vehicle (V2V) communication or vehicle-to-infrastructure(V2X) communication, and LIDAR communications may occur between anycompatible vehicles and/or infrastructure (such as within a fleet ofvehicles or between unrelated vehicles including configured LIDARdevices). LIDAR communications may be used for a variety ofcircumstances and in a variety of use cases, such as described herein.

The following description is directed to certain implementations for thepurposes of describing the innovative aspects of this disclosure.However, a person having ordinary skill in the art will readilyrecognize that the teachings herein can be applied in a multitude ofdifferent ways. The described implementations may be implemented in anydevice, system, or vehicle including or coupled to one or more LIDARdevices. In some implementations, a “device” for performing operationsdescribed herein may refer to a control device or system coupled to avehicle and one or more LIDAR devices; a vehicle including a controldevice or system and coupled to one or more LIDAR devices; a controldevice or system coupled to infrastructure or another non-vehiclesystem; or another suitable implementation. Similarly, a “vehicle” mayrefer to a control device or system coupled to the vehicle; the vehicleseparate from a control device or system coupled to the vehicle; thecombination of the control device or system and the coupled vehicle; oranother suitable implementation.

In the following description, numerous specific details are set forth,such as examples of specific components, systems, and processes, toprovide a thorough understanding of the present disclosure. Also, in thefollowing description and for purposes of explanation, specificnomenclature and/or details are set forth to provide a thoroughunderstanding of the example embodiments. However, it will be apparentto one skilled in the art that these specific details may not berequired to practice the example embodiments. In other instances,well-known circuits, systems, and devices are shown in block diagramform to avoid obscuring the present disclosure. Any of the signalsprovided over various buses described herein may be time-multiplexedwith other signals and provided over one or more common buses.Additionally, the interconnection between components or software blocksmay be shown as buses or as single signal lines. Each of the buses mayalternatively be a single signal line, and each of the single signallines may alternatively be buses, and a single line or bus mightrepresent any one or more of a myriad of physical or logical mechanismsfor communication between components. Additionally, the term “coupled”as used herein means coupled directly to or coupled through one or moreintervening components or devices.

FIG. 1 is a block diagram of an example light detection and ranging(LIDAR) device 100. The LIDAR device 100 may be used to detect objectsurfaces in an environment by emitting pulses of lights that illuminatethe object surfaces and by detecting light pulses reflected from theobject surfaces. The LIDAR device 100 can determine the distance to anobject based on the time delay between the emission of a light pulse andthe reception of a corresponding light pulse reflected from the selectedobject. This time delay, which may be referred to as the ToF of thelight pulse, may be multiplied by the speed of light to determine thedistance between the LIDAR device 100 and the object. Multiple pulsesmay be used to determine distance information of a number of pointsassociated with the object in the environment. The points may be used togenerate a point cloud or otherwise may be used to determine thelocation, size, shape, pose, and motion of the object. In someimplementations, information from the LIDAR device 100 may be used tocontrol an autonomous vehicle, for example, so that the autonomousvehicle can navigate the environment to reach a destination whileavoiding obstacles. The LIDAR device 100 may also be used to measuredistances for driver assistance operations.

The LIDAR device 100 is shown to include a transmitter 110, a receiver120, and a LIDAR controller 130. The transmitter 110 may include atransmit controller 111, one or more light emitters 112, and a transmitaperture 113. The light emitters 112 may emit one or more light pulses125 that can be used to detect objects in a surrounding environment. Thelight emitters 112 may include any number of suitable light sources suchas (but not limited to) laser diodes, light emitting diodes (LEDs),vertical cavity surface emitting lasers (VCSELs), organic light emittingdiodes (OLEDs), polymer light emitting diodes (PLEDs), light emittingpolymers (LEPs), liquid crystal displays (LCDs), microelectromechanicalsystems (MEMS), or any other device configured to selectively transmitor emit light pulses 125 at a defined wavelength. The source wavelengthmay include, for example, the ultraviolet, visible, and/or infraredportions of the electromagnetic spectrum. In some aspects, the lightemitters 112 may be disposed on one or more substrates (e.g., printedcircuit boards (PCB), flexible PCBs, and the like). Although the lightemitters 112 are described herein as emitting light pulses 115, one ofordinary skill in the art will readily understand that the lightemitters 112 may transmit or emit light signals, light beams, photons,and the like. Thus, the terms light pulses, light signals, light beams,and photons may be used interchangeably herein.

The transmit aperture 113 is coupled to the light emitters 112, and mayinclude any suitable components (e.g., mirrors, lenses, diffractiongratings, exit apertures, and the like) that can focus, direct, and/orcondition the light pulses 115 for emission into the surroundingenvironment. In some implementations, the transmit aperture 113 may beconfigured to steer the light pulses 115 in one or more specifieddirections relative to the LIDAR device 100. The specified directionsmay span a range of directions, for example, so that distances betweenthe LIDAR device 100 and a number of objects (e.g., vehicles, people,roads, traffic signals, traffic signs, obstacles, etc.) may bedetermined based on reflections of the light pulses 115 caused by theobjects.

The transmit controller 111 may control operations of the light emitters112 and the transmit aperture 113, may adjust a number of parameters orsettings of the light emitters 112 and the transmit aperture 113, orboth. In some implementations, the transmit controller 111 may beresponsive to one or more control signals provided by the LIDARcontroller 130. For example, the transmit controller 111 may adjust thewidth, the timing, the frequency, and/or the amplitude (intensity) ofthe light pulses 115 emitted by the light emitters 112 based on the oneor more control signals. In other implementations, the transmitcontroller 111 may be omitted or may be included within the LIDARcontroller 130.

The receiver 120 may include a number of photodetectors 121, a detectorcircuit 122, and an analog-to-digital converter (ADC) 123. Thephotodetectors 121 may receive light pulses 125 (e.g., photons) from thesurrounding environment. In some implementations, the received lightpulses 125 may include components of the emitted light pulses 115reflected from one or more objects in the surrounding environment. Thephotodetectors 121 may be configured to convert the received lightpulses 125 into photodetector signals (e.g., analog current signals)indicative of intensity levels of the received light pulses 125. Thephotodetectors 121 may be any suitable component or device that canreceive or sense light including, for example, photodiodes (such asavalanche photodiodes), Silicon Photomultipliers (SiPMs),phototransistors, cameras (such as CMOS sensors), active pixel sensors(APS), charge coupled devices (CCD), cryogenic detectors, or the like.In some implementations, the photodetectors 121 are reverse-biasedphotodiodes that generate a current in response to receiving lightpulses, for example, such that the amount of current through eachphotodiode is proportional to the intensity of light pulses received bythe photodiode.

Although not shown for simplicity, the receiver 120 may include opticsto filter wavelengths of the received light so that the photodetectors121 primarily receive light corresponding to the wavelength of the lightpulses 115 emitted by the transmitter 110 (and receive minimal lightcorresponding to other wavelengths). For example, the receiver 120 mayinclude a bandpass filter to filter optical signals outside of a rangeof wavelengths centered around a base wavelength of the light pulsesemitted by the transmitter 110.

The detector circuit 122 may use any suitable technique to samplephotodetector signals provided by the photodetectors 121 to determineintensity levels of the received light pulses 125. In someimplementations, the detector circuit 122 may sample the photodetectorsignals at a number of intervals or sampling times. In otherimplementations, the detector circuit 122 may continuously sample thephotodetector signals. The detector circuit 122 may provide thedetermined intensity levels to the ADC 123, for example, as analogsignals having a magnitude (e.g., a voltage magnitude or a currentmagnitude) indicative of light information contained in thephotodetector signals. In some aspects, the detector circuit 122 mayamplify and/or filter the photodetector signals.

The ADC 123 may receive analog signals indicating intensity levels ofthe received light pulses 125 from the detector circuit 122, and mayconvert the analog signals into digital data that can be processed bythe LIDAR controller 130. The ADC 123 may be any suitable ADC such as(but not limited to) a flash ADC, a successive-approximation-register(SAR) ADC, or a delta-sigma ADC. In some implementations, eachphotodetector 121 may correspond to a respective ADC. In otherimplementations, a plurality of photodetectors 121 may correspond to asingle ADC (e.g., to reduce the size, cost, and/or power consumption ofthe LIDAR device 100). In some other implementations, the ADC 123 may beomitted. Each of the photodetectors 121 (and corresponding ADCs 123) maybe associated with a specific light emitter 112. In this manner,multiple ToFs may be measured, and therefore multiple distances may bedetermined during one pass of the LIDAR device 100.

The LIDAR controller 130 may include a processor 131, a memory 132, anda digital signal processor (DSP) 133. The DSP 133 may process digitaldata provided by the one or more ADCs 123 to determine informationregarding the light pulses received by any number of the photodetectors121. In some implementations, the intensity and/or time of arrival ofthe light pulses may be used to determine the size, shape, and locationof a number of detected objects in the surrounding environment. Forexample, with the time of departure of a light pulse known, the DSP 133may use the time of arrival to determine the ToF. In another example,with the intensity of the emitted light pulse known, the DSP 133 may usethe measured intensity to determine an energy loss of the reflectedlight pulse. Objects that are relatively close to the LIDAR device 100may reflect emitted light pulses 115 before objects that are relativelyfar from the LIDAR device 100. In addition, light reflected from objectsthat are relatively close to the LIDAR device 100 may have less pulsespreading than light reflected from objects that are relatively far fromthe LIDAR device 100 (assuming similar surface reflectivity between theobjects at varying distances). Thus, in some implementations, thedistance between the LIDAR device 100 and an object may be estimatedbased on the rising and falling edges of the received light pulses 125.

The processor 131 may be any suitable one or more processors capable ofexecuting scripts or instructions of one or more software programsstored in the LIDAR device 100 (e.g., within the memory 132). In someimplementations, the processor 131 may include one or moremicroprocessors and memory providing at least a portion ofmachine-readable media within which program instructions or scripts canbe stored. In other implementations, the processor 131 may be anApplication Specific Integrated Circuit (ASIC). In some otherimplementations, the processor 131 may be or include one or more FieldProgrammable Gate Arrays (FPGAs) or Programmable Logic Devices (PLDs).

The memory 132 may store information pertaining to the transmitter 110,the receiver 120, the surrounding environment, or any combinationthereof. The memory 132 may also include a non-transitorycomputer-readable medium (e.g., one or more nonvolatile memory elements,such as EPROM, EEPROM, Flash memory, a hard drive, and so on) that maystore a number of software (SW) modules each including instructionsthat, when executed by the processor 131, causes the LIDAR device 100 toperform all or a portion of the operations described herein. In someother implementations, the LIDAR controller 130 may be instructed by oneor more processors outside the LIDAR device 100 (such as included in avehicle's processing system coupled to the LIDAR device 100) to performone or more operations regarding the emission or reception of lightpulses. For example, the LIDAR device 100 may be coupled to a processinghub of a vehicle (such as via a Controller Area Network (CAN) Bus) oranother processing system of the vehicle, and the processing system mayinstruct the LIDAR device 100 to perform one or more operations andreceive information from the LIDAR device 100 in response (such asretrieving ToF or intensity information measured by the LIDAR device 100and used by the processing system to generate a point cloud or othertype of depth map or rendering of the environment).

FIG. 2A illustrates an example LIDAR device in an environment. In theexample of FIG. 2A, the LIDAR device 100 is situated in an environment200 including an automobile 201 and an overhang 202. In a simplifiedexample, the LIDAR device 100 is shown to include three light emitters112A-112C that emit respective light pulses 115A-115C into theenvironment 200. The LIDAR device 100 is also shown to include threephotodetectors 121A-121C that receive respective light pulses 125A-125Creflected from surfaces of objects in the environment 200. A first lightpulse 115A illuminates a surface 205A of the overhang 202, and a firstphotodetector 121A receives a corresponding reflected light pulse 125A.A second light pulse 115B illuminates a surface 205B of the automobile201, and a second photodetector 121B receives a corresponding reflectedlight pulse 125B. A third light pulse 115C illuminates another surface205C of the automobile 201, and a third photodetector 121C receives acorresponding reflected light pulse 125C. The LIDAR device 100 may useone or more properties of the received light pulses 125A-125C (e.g.,timing, amplitude, pulse width, and so on) to determine the distancebetween the LIDAR device 100 and each of the surfaces 205A-205C in theenvironment 200.

FIG. 2B is an example timing diagram 210 of waveforms corresponding toemitted light pulses and received light pulses of an example LIDARdevice. Transmit waveforms 215A-215C may be indicative of intensitylevels of respective light pulses 115A-115C emitted from the LIDARdevice 100 of FIG. 2A, and receive waveforms 225A-225C may be indicativeof intensity levels of respective light pulses 125A-125C received by theLIDAR device 100 of FIG. 2A. The light pulses 115A-115C are emitted fromthe LIDAR device 100 at the same time to (or at least substantially thesame time), and the reflected light pulses 125A-125C are received by theLIDAR device 100 at different times t_(A)-t_(C) (e.g., due to differentdistances between the LIDAR device 100 and each of the surfaces205A-205C of the environment 200). The transmit waveforms 215A-215Cinclude respective pulses 216A-216C that represent the time to at whichcorresponding light pulses 115A-115C are emitted from the LIDAR device100. The receive waveforms 225A-225C include respective pulses 216A-216Cthat represent the times t_(A)-t_(C) at which corresponding reflectedlight pulses 125A-125C are received by the LIDAR device 100. Thereception times t_(A)-t_(C) may be determined from respective pulses226A-226C using any suitable peak detection technique (e.g., determininga peak amplitude, determining a centroid, determining a mean timebetween threshold crossings, and the like). The determined receptiontimes t_(A)-t_(C) may be used to determine distances between the LIDARdevice 100 and respective surfaces 205A-205C of the environment 200.

Many LIDAR devices allow adjusting one or more features of the emittedlight pulses. For example, a LIDAR device 100 (FIG. 1) may configure thetransmitter 110 to adjust one or more of the timing of the light pulses,the frequency of the light pulses, or the intensity of the light pulsesfor the emitted light 115. In some examples, the LIDAR device 100 allowsdithering of the light pulses for the emitted light 115 to control thetiming. Some LIDAR devices may allow adjusting the power provided to thelight emitters 112 (such as LEDs) to control the intensity. Some LIDARdevice may allow adjusting a reference signal frequency (such as thelight source) to adjust the frequency of the light pulses of the emittedlight 115. The receiver 120 may also be configured to sense differencesin timing, frequency, or intensity of the light pulses for the receivedlight 125. For example, the sensitivity of a photodiode array for thephotodetectors 121 may be sufficient to determine differences inintensities, and the sampling rate of the photodiode array and/or theADCs may be sufficient to determine differences in timing or frequencyof the light pulses for the received light 125.

In some aspects, a LIDAR device 100 (such as the LIDAR controller 130)may be configured to encode information into the emitted light 115 viafrequency adjustment, intensity adjustment, and/or time dithering of thepulses to be emitted, and the transmitter 110 may be configured tocommunicate such encoded information via the emitted light 115. Forexample, the LIDAR device 100 may receive a data signal to betransmitted, and the LIDAR device 100 may use the light to be emitted(which may be referred to as an optical signal herein) as a carriersignal. In this manner, the LIDAR device 100 may modulate the opticalsignal to include the data signal, and the LIDAR device 100 may transmitthe modulated optical signal. For a first LIDAR device, the receiver 120may be configured to receive a modulated optical signal transmitted by atransmitter 110 of a second LIDAR device. The modulated optical signalis encoded with a data signal from the second LIDAR device, and theLIDAR controller 130 of the first LIDAR device may be configured toextract the data signal from the received modulated optical signal. Forexample, the LIDAR controller 130 may demodulate the received opticalsignal to generate the data signal. Since such communication betweenLIDAR devices is point-to-point (and thus not requiring a centralnetwork) and the frequency of the signals is greater than for cellularcommunications, the throughput may be higher and the latency may belower than for traditional cellular communications. A “modulated opticalsignal” herein refers to an optical signal (such as light emitted by aLIDAR device 100) modulated to include a data signal.

In some implementations, the LIDAR device 100 may be configured toswitch between a distance measurement mode (which may be called a“detection mode” herein) and a communication mode for transmitting amodulated optical signal including a data signal. For example, the LIDARcontroller 130 may determine when the LIDAR device 100 is to transmit orreceive information via a modulated optical signal and when the LIDARdevice 100 is to operate in detecting surfaces of objects in anenvironment. For example, the LIDAR controller 130 may determine toplace the LIDAR device 100 in a communication mode for a first portionof time and place the LIDAR device 100 in a detection mode for a secondportion of time. The LIDAR device 100 may therefore switch betweencommunication and detection modes. In some other implementations,emitted light 115 may be used for communicating information to anotherreceiver (such as via adjusting the frequency) and also used fordetecting surfaces of objects (such as via sensing differences inintensities in the received light 125), and the LIDAR device 100 may beconfigured to perform both modes concurrently. For example, a LIDARdevice 100 may emit a modulated optical signal (including a datasignal). The LIDAR device 100 may receive reflections of the modulatedoptical signal and use the reflections for determining depths of objectsfrom the LIDAR device 100. Additionally, a second LIDAR device mayreceive the modulated optical signal and demodulate the optical signalto generate the included data signal from the LIDAR device 100.

A device, such as a vehicle or infrastructure, may include or be coupledto one or more LIDAR devices (such as the LIDAR device 100) configuredto communicate a data signal (provided by, e.g., the vehicle orinfrastructure) via the emitted light 115. In this manner, the devicemay communicate with vehicles or infrastructure using the configuredLIDAR device. In some aspects, the LIDAR device 100 may be configured touse a communication protocol adopted by other vehicles andinfrastructure including LIDAR devices. The communication protocol maybe ad-hoc or managed, and any suitable packetization of information maybe used for the communication protocol. For example, LIDARcommunications within a fleet of vehicles may be based on a specificprotocol. In some implementations, a standardized protocol (or aprotocol adopted by multiple parties) may more easily allow integrationof LIDAR communications between vehicles and infrastructure. Such aprotocol may include a defined packet format for transmitting andreceiving information. Protocols from other communication mediums, suchas cellular communications, Wi-Fi communications, digital subscriberline (DSL) communications, fiber optic communications, etc., may beleveraged in creating a protocol for communications between LIDARdevices.

FIG. 3 illustrates an example packet format for a plurality of receivedpackets 300 based on an example communication protocol. In someimplementations, the LIDAR device 100 may include a buffer or othersuitable memory (such as memory 132) for queuing one or more receivedpackets 300 (such as the packets 1-6). The LIDAR device 100 may processthe buffered packets for the vehicle or infrastructure to perform one ormore operations. In another example, the buffered packets may beprovided to a processing system of the device for processing. Asillustrated, the format for packets 1-6 includes fields 302-308, whichincludes a transmitter ID 302, a location 304, a payload type 306, and apayload 308. Additional fields, fewer fields, different fields, or adifferent organization of fields may be included in the packets, as anysuitable packet format may be used. For example, the packets 1-6 mayinclude a cyclic redundancy check (CRC) field after the payload 308 tocorrect any errors in the received packet.

The transmitter ID 302 may indicate the device transmitting the packet.In some implementations, each vehicle may include a unique identifier toidentify that specific vehicle. For example, if a fleet of 200autonomous vehicles communicate with one another, each vehicle mayinclude an identifier unique from the other vehicles in the fleet (suchas “vehicle_1” through “vehicle_200”). If a LIDAR device equipped on avehicle associated with “vehicle_100” transmits packets 1 and 2 to theLIDAR device 100, the transmitter ID 302 of the packets may include theunique identifier “vehicle_100.” Similarly, if a LIDAR device equippedon a vehicle associated with “vehicle_102” transmits packet 3 to theLIDAR device 100, the transmitter ID 302 of the packet may include theunique identifier “vehicle_102.” Some vehicles may not include a uniqueidentifier or otherwise not be identified, but the vehicles may stillcommunicate packets to the LIDAR device 100. In one example, thetransmitter ID 302 may include a transmitter ID value specific topreviously unidentified vehicles or devices (illustrated as“Unidentified_Vehicle” for packet 4). The transmitter ID value may be,e.g., zero padding, a field with null values, or otherwise suitablyfilled to indicate no unique identifier for the vehicle orinfrastructure associated with the packet.

Infrastructure (such as toll booths, tunnels, entry gates for highoccupancy vehicle (HOV) lanes, etc.) may include or be coupled to one ormore LIDAR devices to transmit and receive packets to and from the LIDARdevice 100. For example, packet 5 may be transmitted by a transmitterpositioned at the entrance of a tunnel (which may be uniquely identifiedas “Tunnel_40”), and packet 6 may be transmitted by a transmitterpositioned at a toll booth (which may be uniquely identified as “TollBooth_30”). Any suitable vehicle or infrastructure may include a LIDARdevice for LIDAR communications and may include a unique transmitter ID.

In some implementations, emergency vehicles, construction zones, andother devices associated with priority transmissions may include aTransmitter ID indicating that the packet is for a prioritytransmission. If the LIDAR device 100 decodes the packet sequentially,the LIDAR device 100 may process the transmitter ID 302 first anddetermine that the packet is a priority transmission. In this manner,the LIDAR device 100 may determine that the packet is to be prioritizedbefore completing processing the remainder of the packet. Other suitableimplementations of a transmitter ID 302 may be used, and the presentdisclosure is not limited to the provided examples. For example, atransmitter ID may be configured to differentiate between fleets,vehicle types (such as personal cars, commercial trucks, school buses,etc.), etc.

The location 304 may indicate the location of the vehicle orinfrastructure associated with the transmitting device. For example, avehicle or infrastructure may include a global positioning system (GPS)receiver to determine a latitude and longitude. The latitude andlongitude may then be provided in the location 304, such as illustratedfor packets 1-6. Location 304 may alternatively include positioninginformation relative to the LIDAR device 100, such as determined by theother vehicle or infrastructure via its own LIDAR device. For example,the transmitting LIDAR device may determine a relative distance andposition between the transmitting and receiving vehicles orinfrastructure, and the device distance and position may be transmittedin the location 304. In some implementations, if location information isnot available to send (such as a vehicle does not include a GPSreceiver), the location 304 may be padded with zeros or otherwise filledto indicate no location is provided.

The payload type 306 may indicate the type of information to be providedin the payload 308 of the packet. In the example packets 1-6, packet 1'spayload type 306 indicates a “HEARTBEAT” or “BEAT” which may be a signalperiodically transmitted by “Vehicle_100.” In some implementations, a“HEARTBEAT” or “BEAT” may indicate that the packet is similar to abeacon, and base information may be provided in the payload 308. In someexamples of providing base information of the vehicle or infrastructure,the payload 1 may include a dither rate or other operating parametersfor the transmitter and/or trajectory information or other stateinformation for a vehicle. Alternatively, no information may be providedin the payload 308 for such packets (e.g., the payload is zero paddedand the packet is only for providing location information via location304 and to apprise other devices of the existence of the transmitter).

The payload type 306 for packet 2 is “RENDER_DESCRIPTION.” As describedin example use cases below, a LIDAR device 100 may measure surfaceswithin the line of sight (LoS) of the LIDAR device 100, but surfaces notwithin the LoS of the LIDAR device 100 may not be sensed and detected.In this manner, a rendering of the environment based on measurementsfrom the LIDAR device 100 may not include portions of a vehicle notwithin view of the LIDAR device 100. The “RENDER_DESCRIPTION” mayindicate that basic information regarding rendering the vehicle orinfrastructure may be provided in the payload 308. For example, payload2 may include dimensions of one or more shapes associated with“Vehicle_100” to be rendered. In one implementation, the dimensions mayinclude an orientation and dimensions of a geometric shape (such as arectangle for two-dimensional rendering or a rectangular prism for threedimensional rendering). Other suitable rendering information may includethe texture, outline, or other features of the object's renderingindicated by the payload 308. For example, an emergency vehicle or aroad construction zone may be highlighted in a rendering displayed by avehicle to a driver and/or passenger. The payload 308 may thus indicatea texture to highlight the portion of the rendering associated with theemergency vehicle or construction zone. In another exampleimplementation, a similar texture may be used for vehicles within thesame fleet. In this manner, a driver and/or passenger may easilyidentify fleet vehicles in a displayed rendering. Any suitable renderingdescription may be used, though, and the present disclosure is notlimited to the provided examples.

Additional or alternative to a “RENDER_DESCRIPTION,” the payload type306 may indicate a “RENDER_IDENTIFICATION,” such as for packet 4. Insome implementations, a list of standard vehicles (or infrastructure)and their associated rendering information may be stored in a database(such as a look-up table or another suitable set of data organized foraccess). For example, if the vehicle transmitting the packet is a 1994Geo Metro, the payload 308 (such as payload 4) may indicate a “1994 GeoMetro.” The database may be used in determining the rendering detailsfor the specific vehicle (such as the dimensions and textures forrendering), and an existing rendering may be enhanced by the additionalrendering details. The database may also include information about thespecific vehicle, e.g., length, acceleration capability, etc. The term“rendering,” herein, may mean line drawings, point clouds, depth maps,images, texture, shading, or other information that may be visualized ordisplayed.

The payload type for packet 5 is “GATED-ENTRY,” which may indicate thatpayload 5 indicates the type of vehicles allowed to enter “Tunnel_40.”For example, “Tunnel_40” may be limited to high occupancy or autonomousvehicles, which may be indicated in packet 5. The vehicle including theLIDAR device 100 may thus determine whether entry into “Tunnel_40” ispermitted based on the information in payload 5.

The payload type of packet 6 is “FEE_CHARGE,” which may indicate thatpayload 6 indicated the fee charged by “Toll Booth_30.” For example, ifa bridge toll is five dollars, payload 6 may indicate that five dollarsis to be automatically collected from a driver or fleet's account whenthe vehicle passes “Toll Booth_30.” In some implementations, the payload308 may include a combination of fees to be charged and limitations onvehicle entry. For example, a transmitter for a parking garage space mayindicate the rate to park as well as the allowed vehicles in the space(e.g., reserved for electric vehicle space, designated handicappedparking, etc.). While the vehicle is parked in the space, thetransmitter may update the fees for parking based on the rate so that adriver may be aware of the current parking costs.

The payload type of packet 3 is “VEHICLE_LOCATOR,” which may indicatethat payload 3 indicates a vehicle that is attempting to be found. Forexample, if “Vehicle_005” is offline, the vehicles in the fleet may sendpackets attempting to locate “Vehicle_005.” In this manner, payload 3may include the “Vehicle_005” identifier to indicate which vehicle is tobe located. In some implementations, the LIDAR device 100 may persist anumber of transmitter IDs of packets received. The LIDAR device 100 maytherefore be configured to search the persisted transmitter IDs ofpackets received to determine if “Vehicle_005” communicated with theLIDAR device 100. The LIDAR device 100 may then indicate such to“Vehicle_102.” For example, the LIDAR device 100 may transmit thelocation received in the packet from the “Vehicle_005” to the“Vehicle_102.” In some other implementations, the LIDAR device 100 maybe configured to propagate the message to other vehicles orinfrastructure, thus indicating that “Vehicle_005” is to be located. Inthis manner, if “Vehicle_005” receives the propagated message fromanother vehicle or infrastructure, the “Vehicle_005” may determine tocommunicate with the fleet its location.

Other suitable payload types and payloads may exist (such as fordifferent use cases for LIDAR communication as described herein), andthe present disclosure is not limited to the provided examples. In someexample implementations, a vehicle may include a plurality of sensors tocollect information for packet generation. For example, a payload mayinclude information regarding the number of passengers, the operatingmode of the vehicle, etc., and sensors, such as pressure sensors,occupancy sensors, engine sensors to detect the operating mode, etc. maybe used to collect such information.

A diverse range of information may be provided and received via LIDARcommunications, and LIDAR communications may apply to a variety of usecases, such as described herein.

LIDAR Communication Implementations

Vehicle Locator

A vehicle may operate in areas without a cellular network forcommunication. For example, when a vehicle crosses rural or sparselypopulated areas, the vehicle may not have a consistent cellularconnection or otherwise be able to communicate with a base station. Inanother example, the vehicle's cellular modem may be inoperable. If thevehicle is part of a fleet of vehicles (such as a taxi or rental carfleet), a dispatcher may be unable to identify the location of thevehicle, and the dispatcher may wish to find the vehicle.

In some implementations, searching for a missing, stranded, or offlinevehicle may be proactive, where a vehicle or infrastructure finding anoffline vehicle reports back to the dispatcher that the offline vehicleis found. In some other implementations, searching for an offlinevehicle may be reactive, where a vehicle logs the last known location ofthe offline vehicle. In this manner, the vehicle reports its findingswhen syncing with the dispatcher or other portions of the fleet (such aswhen the vehicle is charging or otherwise not in operation).

Other vehicles in the fleet may be instructed to transmit a VehicleLocator message via LIDAR communication. For example, each of the fleetvehicles may be instructed to periodically transmit a packet with apayload type of “Vehicle_Locator,” identifying the offline vehicle inthe packet payload. The dispatcher may also instruct infrastructure,such as fleet owned electric vehicle charge stations or parking spacesthat include or are coupled to a LIDAR device, to transmit VehicleLocator messages. As described above, other vehicles may also propagatesuch messages once received to expand the number of transmitters sendingthe Vehicle Locator message. In this manner, the vehicle, which may beoffline relative to its primary communication method, e.g., cellular,may receive the message from a passing vehicle or infrastructure, andthe vehicle may determine to communicate its location to the dispatchervia the LIDAR communication. In another example, the vehicle orinfrastructure finding the offline vehicle may communicate the offlinevehicle's location to the dispatcher.

For example, if another fleet vehicle is in LIDAR communication with theoffline vehicle, the offline vehicle may transmit a message to the otherfleet vehicle via LIDAR communication that its cellular modem isinoperable and to, e.g., contact the dispatcher via the other vehicle'scellular modem. In this manner, the dispatcher may communicate with theoffline vehicle via the other fleet vehicle.

In addition or alternative to a dispatcher finding an offline fleetvehicle, Vehicle Locator messages may be used in emergency situations,such as for stolen vehicles. A payload of a Vehicle Locator packet mayinclude a description of the vehicle to be found, and such packet may bepropagated to other vehicles and infrastructure to find the stolenvehicle. In some implementations, a vehicle may display a notificationto the driver in response to receiving the packet. In some otherimplementations, the stolen vehicle may be automatically identifiedbased on the dimensions or other properties of the stolen vehicle. Forexample, detected surfaces of other vehicles using a LIDAR device may beused to identify a vehicle with the same make and model based on similardimensions or other features of the stolen vehicle. In another example,a visible light camera may be used to capture an image of a licenseplate after detecting a missing vehicle, and the image may be analyzedto identify the missing vehicle based on the license plate number. Thevehicle may report (such as via a cellular modem) the location of themissing vehicle to a central office.

In some other implementations, a Vehicle Locator message may beinitiated by an offline or disabled vehicle. For example, a fleetvehicle's cellular modem may become inoperable, and the fleet vehicle isunable to communicate with the fleet via a cellular network. If thedispatcher provides information about where the fleet vehicle is to govia cellular communications, the fleet vehicle may not be able todetermine where the dispatcher wishes to send the vehicle. In theexample of an autonomous vehicle taxi fleet, the dispatcher may wish tosend the vehicle to pick up a client at a specific address and taxi theclient to a specific destination. However, the vehicle is unable toreceive the instructions from the dispatcher. LIDAR communications maybe used to provide the dispatcher with any messages from the vehicle(such as the vehicle having a disabled cellular modem). For example, thevehicle may transmit, to other fleet vehicles via LIDAR communications,messages intended for the dispatcher. The other fleet vehicles may thentransmit the messages to the dispatcher. In this manner, the dispatcheris apprised that the vehicle may not be able to receive messages fromthe dispatcher, and the dispatcher may update handling requests withoutthe vehicle in the fleet. The dispatcher may also provide instructionsto the vehicle via LIDAR communications (such as via other fleetvehicles) to wait at a designated spot for service, remove itself fromthe fleet's operation, or any other suitable operations.

In some implementations, the vehicle may have a home location away fromthe taxi fleet dispatcher. For example, if the vehicle does not have alocation to go to (such as for a fare communicated by the dispatcher),the vehicle may return to a designated spot in a service area, such as aspecific parking space for the vehicle within the service area. Thespace may be associated with a LIDAR device that may communicate withthe vehicle (allowing, e.g., the dispatcher to communicate with thevehicle). When the vehicle is fully operable and back in standardcommunication with the dispatcher, the designated spot may be a holdingarea for the vehicle until the vehicle is activated for a fare. When thevehicle is unable to communicate with the dispatcher and is not in usefor a fare, the vehicle may also return to the designated spot (such asthe parking space). References to offline vehicles herein may refer tostranded, missing or other vehicles that are not operating or may not beexpected to operate within a default, typical, or standard operatingmode.

FIG. 4 illustrates an environment 400 including an example parking space410 for an autonomous fleet vehicle 402 to return to when not in use.The autonomous vehicle 402 includes one or more LIDAR devices, such asthe LIDAR device 404. A stand 408 may be situated near the parking space410. Examples of the stand 408 include a parking meter, a chargingstation, and a taxi stand for walk-up clients. In some implementations,the stand 408 includes a LIDAR device to communicate with the vehicle402 via LIDAR communications 406. The stand 408 also includes a wired orwireless backhaul (such as a cellular modem or fiber-optic connection).The vehicle 402, which is unable to communicate directly with thedispatcher, may communicate with the stand 408 via LIDAR communications,and the stand 408 may communicate with the dispatcher via the backhaul.While the vehicle 402 is described as communicating with the stand 408via LIDAR communications, additional or alternative communicationssystems may be used in some other embodiments. Other systems may includewireless local area network systems (such as IEEE 802.11 based systems),BLUETOOTH® systems, cellular systems (such as 3G, 4G, 5G, etc.), visiblelight communication systems, near field communication (NFC) systems,etc.

In communicating with the stand 408, the vehicle 402 may provide aVehicle Locator packet to the stand 408. In some implementations,transmitting the Vehicle Locator packet may indicate to the stand 408and the dispatcher that the vehicle 402 is unable to communicatedirectly with the dispatcher (such as via a cellular modem). In someexamples, the dispatcher may remove the vehicle 402 from service, send amaintenance crew to the vehicle 402, send a location for the vehicle 402to go for maintenance, or communicate a new fare to be handled by thevehicle 402. In some implementations, the stand 408 may also communicatewith the vehicle 402 while passing on the road. If the vehicle 402 isunable to communicate directly with the dispatcher, the stand 408 mayupdate the dispatcher with a location of the vehicle 402.

Vehicle Hailing

In addition to locating a vehicle, LIDAR communication may be used forhailing a vehicle. Referring back to FIG. 4, the stand 408 may be awalk-up taxi stand. A person may walk-up to the stand 408 and enter adesired destination (such as via a smartphone application, a graphicaluser interface of the stand 408, or another suitable interface with thestand 408). A plurality of automated vehicles may be queued at alocation away from the stand 408 (such as in a parking lot around thecorner) and in communication with another LIDAR transmitter (such as oneor more stands in the parking lot). The parking space 410 may be thelocation where the next vehicle in the queue is hailed, and the parkingspace 410 may be where the user enters the vehicle and embarks on therequested ride.

The stand 408 may handle multiple requests for rides concurrently. As aresult, the stand 408 may hail a plurality of vehicles (such as usingone or more LIDAR transmitters in the staging area for the queuedvehicles). Each hailed vehicle may be assigned a specific client orride, and when the vehicle approaches the parking space 410, the vehiclemay indicate to the stand 408 (via LIDAR communication 406) the rideassigned. In this manner, the stand 408 may notify the clients of theride assigned to the vehicle in the parking space 410 to expediteembarkation and coordinate rides for multiple clients from the sameparking space 410.

Notifications

LIDAR communication may be used to provide notifications to vehicles ordrivers. For example, a vehicle may be notified of, e.g., an approachingemergency vehicle, a road hazard, a school crossing during school hours,changes in the speed limit, a construction zone, etc., via LIDARcommunication. For emergency vehicles, an emergency broadcast may becommunicated by infrastructure and vehicles previously receiving thebroadcast to notify other vehicles of the presence of an emergencyvehicle via LIDAR communication.

FIG. 5 illustrates an example environment 500 for communicating thepresence of an ambulance 502 to the vehicle 506. The vehicle 508 maydetect the ambulance 502. For example, the ambulance 502 may broadcastan emergency signal indicating the presence of the ambulance 502. Inanother example, a LIDAR device or other sensors of the vehicle 508 maysense the ambulance 502, and the vehicle 508 may identify the ambulance502 as an emergency vehicle.

In some examples other than an ambulance 502, the vehicle 508 may detectpersons in the environment that may warrant other vehicles to altertheir navigation or be notified of such persons. For example, thevehicle 508 may detect one or more bicyclists in a lane, childrenplaying near the street, children or other pedestrians crossing thestreet (such as before or after school), etc. In other examples, thevehicle 508 may detect a stalled car, minor traffic accident (“fenderbender”), road blockage, or other traffic situation that may warrantother vehicles altering their navigation or be notified of suchaccident. Such scenarios may warrant that vehicles reduce their speedthrough the area, adjust scanning the environment using one or moreLIDAR devices to focus on specific areas, change lanes, e.g., to providemore space to the detected persons, situation, etc.

The vehicle 508 may not be in LIDAR communication range with the vehicle506. For example, LIDAR communications may be based on LoS, and thevehicle 506 may be obstructed or otherwise occluded by a building toprevent the vehicle 508 from communicating directly with the vehicle506. In some implementations, the vehicle 508 may use one or morevehicles and/or infrastructure to perform LIDAR communication with thevehicle 506 (e.g., to communicate “around a corner”). For example, thevehicle 508 may use LIDAR communications 510 with vehicle 504 toindicate the presence of the ambulance 502. In addition, or in thealternative, the vehicle 508 may communicate via LIDAR communications516 with a stand 514 to indicate the presence of the ambulance 502.

While FIG. 5 illustrates direct LoS communications between LIDAR devices(510, 512, 516, and 518), LoS communications between LIDAR devices mayinclude one or more signal reflections off of surfaces. For example, afocused beam of optical signals from a transmitting LIDAR device may beintentionally reflected off one or more building walls, signs, or otherobjects in the environment 500, and the reflections may be received by areceiving LIDAR device. If the beam is sufficiently focused, thetransmission includes a sufficient transmission power, and the one ormore surfaces have a sufficient reflectivity, the reflections receivedby the receiving LIDAR device may be processed to determine thecommunications sent by the transmitting LIDAR device (such as anotification of the presence of the ambulance 502). In this manner,LIDAR devices may communicate with one another without a direct LoS. Inthis specification, LoS may include direct LoS or indirect LoS (whichmay include one or more reflections).

The vehicle 504 or the stand 514 may be configured to indicate thepresence of the ambulance 502 or other traffic situation to othervehicles and infrastructure within LIDAR communication range. Forexample, the vehicle 504 may use LIDAR communications 512 to indicatethe presence of the ambulance 502. In another example, the stand 514 mayuse LIDAR communications 518 to indicate the presence of the ambulance502. In the example, indicating the presence of the ambulance 502 mayinclude two hops to reach vehicle 506 from vehicle 508 (e.g., via LIDARcommunications 510 and 512 or via LIDAR communications 516 and 518).

In some implementations, the indication of an emergency vehicle (orother notifications) may be propagated for a defined number of hops orfor a defined distance. In this manner, notifying other vehicles of thepresence of an emergency vehicle is localized to an area around theemergency vehicle or traffic situation. In some examples, the vehicle508 may also determine a trajectory, route, or other information of theambulance 502 and indicate such information to the vehicle 504 or thestand 514. The distance or number of hops to propagate the indicationmay, e.g., increase in the direction of the route (or, conversely,decrease in the direction opposite of the route). If the vehicle 506 isan autonomous vehicle, the vehicle 506 may proactively pull to the sideof the road, stop, change lanes, or alter its route in response toreceiving the indication of the presence of the ambulance 502 or othertraffic situation. In some other implementations, the vehicle 506 mayprovide a visual or audible notification to the driver (and/orpassengers) to indicate the presence of the ambulance 502 or trafficsituation. For example, a vehicle's speakers or a display may notify thedriver and/or passengers of the presence of the ambulance 502 or trafficsituation.

Another example notification may be of available parking spaces in aparking lot or garage. In some implementations, an entry gate to theparking garage may communicate available spaces to an entering vehiclevia LIDAR communication. If the vehicle is an autonomous vehicle, thevehicle may be assigned a parking space and automatically proceed to andpark in the space. In this manner, the parking garage or lot mayefficiently organize the parking of vehicles entering and exiting. Insome other implementations, the entry gate may indicate availableparking spaces to the vehicle, and the vehicle may notify the driverand/or passengers of the locations of available spaces. For example, thevehicle may display a map of the parking garage and indicate thelocations of the available spaces on the displayed map. The driverand/or passenger may be notified before the vehicle enters the parkinggarage (e.g., as the vehicle approaches the garage), and the driverand/or passenger may select which spaces are preferred.

As noted above, a further example notification may be of a trafficaccident or other obstructions to traffic. The notification may bepropagated via LIDAR communication to vehicles preceding theobstruction, and the vehicles may determine alternate routes and/ornotify the driver and/or passengers of a possible delay based on thereceived notification. Other suitable example notifications may also betransmitted via LIDAR communications, and the present disclosure is notlimited to the above examples.

Limited Access Area

Another use case for LIDAR communications is to limit access of definedareas. Some areas may be limited to specific vehicles or vehicle types.For example, a hospital ambulance entrance may be limited to ambulances.In another example, handicap parking spaces may be limited to vehiclesincluding a handicap tag. In a further example, HOV lanes may be limitedto vehicles including, e.g., 3 or more passengers. In another example,portions of a city center may be limited to autonomous vehicles, lowemission or zero emission vehicles, taxis, or other specific types ofvehicles. In a further example, a company parking garage may be limitedto employee vehicles or vehicles with security clearance.

LIDAR communications may be used to indicate restrictions on specificareas and grant access to such areas for specific vehicles. In someimplementations, a LIDAR device for the area (such as an entry gate,stand, or other infrastructure) may transmit to an approaching vehiclethe restrictions for the area. The vehicle may then communicate (via itsown LIDAR device) the necessary authentication to the LIDAR device,indicating that the vehicle is approved for the area. The vehicle maythen be allowed to enter and navigate the area (such as a gate openingor the vehicle otherwise being granted access to the area). In someother implementations, the vehicle may store the qualifications requiredfor the area. In this manner, the vehicle may preemptively communicateits credentials to a LIDAR device to gain access. For example, as anambulance approaches an ambulance entrance at the hospital, theambulance may use a LIDAR device to identify itself as an ambulance togain entrance.

Access restrictions to an area may change over time. For example, HOVlanes may be restricted only during designated rush hours. In anotherexample, city centers may be restricted to zero emission vehicles whensmog levels are above a threshold. In a further example, areas around anaccident or where police or emergency responders are needed may berestricted to such police or emergency responder vehicles. Since LIDARcommunications do not require a centralized network, changes to accessrestrictions may be more quickly enacted and provided to vehicles than,e.g., remotely coordinating and attempting to establish accessparameters for an area via other communications. In some examples, thevehicle may notify a driver and/or passengers that the vehicle is aboutto enter a restricted area. In this manner, the driver, passengers, orvehicle itself may adjust the route to navigate around the restrictedarea.

In another implementation, areas may include different tolls or chargesbased on the time of day, the day of the week, the amount of congestionin the area, special events, etc. LIDAR communications may be used tocommunicate such tolls or charges to the vehicle before the vehicleenters the area. The driver and/or passengers may then determine (or thevehicle may automatically determine) to enter the area based on theinformation. For example, some roads include hot lanes where the priceto enter such lane is based on the time of day and congestion on theroad. The driver and/or passengers may be notified of the price forentering the hot lane and thus decide whether to enter. Alternatively,the vehicle may be configured to enter the hot lane based on, e.g., ifthe price is less than a threshold amount or a priority of the ride. Forexample, if an expectant couple is travelling to the hospital, thepriority of the ride may be set to override any price constraintsregarding the hot lanes. Other suitable restriction use cases may exist,and the present disclosure is not limited to the provided examples.

Object Visualization or Rendering

Another use case for LIDAR communications is to provide rendering orvisualization information. As described regarding FIG. 3, a packet mayinclude “RENDER_DESCRIPTION” or “RENDER_IDENTIFICATION” information forrendering or visualizing a vehicle (or infrastructure, such as a stand408 in FIG. 4). Rendering or visualizing objects in a vehicle'senvironment may be limited to the LoS of LIDAR devices coupled to thevehicle. As a result, surfaces not in the LoS of the LIDAR devices maynot be detected and rendered.

FIG. 6A illustrates an example environment 600 of the vehicle 602. Theenvironment 600 includes buildings 604 and 606 and vehicles 608 and 610.As illustrated, the view of the vehicle 608 from the perspective of aLIDAR device 612 of the vehicle 602 includes the front and driver sidesurfaces of the vehicle 608. The view of the vehicle 610 from theperspective of the LIDAR device 612 is occluded by the building 606. Asa result, the view of the vehicle 610 includes the front surfaces andonly a portion of the driver side surfaces. Additionally, the view ofthe buildings 604 and 606 from the perspective of the LIDAR device 612are the surfaces facing the vehicle 602. The LIDAR device 612 maycollect information regarding the depths of surfaces within the view ofthe LIDAR device 612, and the vehicle 602 may visualize or render theenvironment 600 including the detected surfaces. A vehicle visualizingor rendering an environment, as used herein, may include one or more of:the vehicle or infrastructure providing, to a display coupled to thevehicle or infrastructure (such as a tablet communicably coupled to aprocessing system of the vehicle, an in-dash entertainment systemcoupled to the processing system of the vehicle, a display remote to thevehicle, etc.), instructions for rendering at least a portion of theenvironment on the display; or the vehicle rendering at least a portionof the environment on an integrated display (such as via an integratedentertainment system in the center console of a vehicle). As usedherein, a specific vehicle or infrastructure performing one or moreoperations (such as a vehicle visualizing or rendering an environment)may refer to a device or system coupled to the vehicle or infrastructureperforming the operations (such as generating a rendering to bedisplayed, and providing the rendering to a display coupled to thevehicle or infrastructure).

FIG. 6B illustrates an example rendering 650 of the environment 600 assensed by the LIDAR device 612. As described above, the examplerendering 650 may be displayed on an integrated display of a vehicle ora display coupled to a processing system of the vehicle. The rendering650 includes renderings of a surface 654 corresponding to the building604, a surface 656 corresponding to the building 606, surfaces 658corresponding to vehicle 608, and surfaces 660 corresponding to vehicle610. The example rendering 650 also includes a representation 652 of thevehicle 602 to provide perspective. For example, the representation maybe an icon, stock image, block, or other saved representation of thevehicle to be used for rendering.

The example rendering 650 is simplified to explain aspects of thedisclosure, and the rendering 650 may include additional information ordetails. For example, the lines or markers in the road may be reflectiveand thus sensed via LIDAR. In another example, additional features ofthe buildings 604 and 606 and vehicles 608 and 610 within the LoS of theLIDAR device 612 may be sensed via LIDAR. While the example rendering650 is illustrated as two dimensional from a bird's eye view, therendering may be any suitable orientation or from any suitableperspective. For example, the rendering may be a three dimensional pointcloud from the perspective of the LIDAR device 612, a two dimensionalview of the environment from the perspective of the LIDAR device 612, orany other suitable rendering.

The example rendering 650 may be provided by the vehicle 602 to a driverand/or passenger via a vehicle display. For example, the examplerendering 650 may be displayed on an integrated display of a vehicle, orthe vehicle may provide instructions for rendering to a display coupledto a processing system of the vehicle. As illustrated in the examplerendering 650, depth information may not be determined for hiddensurfaces of the objects and buildings. For example, the rendering 650does not include information for the passenger side and rear surfaces ofthe vehicle 608, does not include information for the passenger side,rear, and a portion of the driver side surfaces of the vehicle 610, anddoes not include information for sides of the buildings 604 and 606 notdirected to the vehicle 602.

In some implementations, another LIDAR device may transmit to thevehicle 602 rendering information for one or more objects in theenvironment 600. For example, the LIDAR device 614 of the vehicle 608may transmit information regarding the vehicle 608 for rendering, andthe LIDAR device 616 of the vehicle 610 may transmit informationregarding the vehicle 610 for rendering. In some implementations, thedimensions of shapes for rendering a vehicle may be transmitted. Forexample, the dimensions of one or more rectangles for rendering thevehicle 608 in the rendering 650 may be transmitted from the LIDARdevice 614 to the LIDAR device 612. Dimensions of one or more rectanglesfor rendering the vehicle 610 in the rendering 650 may be transmittedfrom the LIDAR device 616 to the LIDAR device 612. Other featuresregarding rendering the vehicle 608 and the vehicle 610 may also betransmitted. Example features include a texture, such as a color, ashading, a highlight, etc. for the vehicle rendering.

In some other implementations, the LIDAR devices 614 and 616 maytransmit an identification of the respective vehicle. For example, theLIDAR device 614 may transmit information about the make and model ofthe vehicle 608, a vehicle identification number of the vehicle 608, alicense plate number of the vehicle 608, whether the vehicle is part ofa specific fleet of vehicles (such as the same vehicle fleet as thevehicle 602), a unique identifier within the fleet of vehicles, or otheridentifying information.

The vehicle 602 (or a memory coupled to a display to display therendering) may store rendering information for specific vehicles, andthe vehicle 602 may use the vehicle identification to look up the storedrendering information for the specific vehicle. For example, a vehiclememory (or other memory) may store rendering information for a pluralityof makes and models of vehicles (including the vehicle 608 and thevehicle 610), and such rendering information may be retrieved and usedto enhance the rendering 650 for the environment 600. If the rendering650 is to be displayed on a display coupled to the vehicle 602, thevehicle 602 may provide the rendering information retrieved from avehicle memory, or the vehicle 602 may provide information foridentifying the rendering information in an external memory (such asproviding information regarding the make and model to an entertainmentsystem coupled to the vehicle). Example rendering information mayinclude a stock image or drawing of the specific vehicle. In someexamples, the rendering information may be used to replace or becombined with corresponding portions of the rendering 650. In some otherexamples, adjusting a rendering may include highlighting an objectrendering. For example, the rendering may be updated to notify a driveror passenger of an emergency vehicle or situation in the vehicle'senvironment.

FIG. 6C illustrates an example environment 670 of the vehicle 672. Theexample environment 670 includes buildings 684 and 686 and vehicles 674and 676. The environment 670 also includes emergency vehicle 678. FIG.6D illustrates an example top down rendering 690 of the environment 670of the vehicle 672 in FIG. 6C. The rendering 690 includes renderings ofa surface 694 corresponding to the building 684, a surface 696corresponding to the building 686, and surfaces 698 corresponding tovehicle 674. The example rendering 690 also includes a representation692 of the vehicle 672 to provide perspective. Vehicle 676 and emergencyvehicle 678 may not be within a LoS of a LIDAR device coupled to thevehicle 672. As a result, the rendering 690 may not include a renderingof the vehicle 676 or a rendering of the emergency vehicle 678.

FIG. 7 shows a flow chart depicting an example operation 700 foradjusting a rendering of an environment of, e.g., a vehicle, a LIDARdevice, an infrastructure, etc. The example operation 700 is describedbelow with respect to the vehicles 602 and 672 in FIGS. 6A and 6C forillustrative purposes only. One of ordinary skill in the art willrecognize that the example operation 700 may be performed by anysuitable device (such as a control device coupled to a vehicle),infrastructure, and/or vehicle according to various implementations, andthe example operation 700 described herein may be performed withadditional steps, with fewer steps, with steps in a different order,with steps in parallel, or any combination thereof. Herein, the vehicle602 performing one or more steps may mean a processing system coupled tothe vehicle performing one or more steps, a processing system integratedinto the vehicle 602 performing one or more steps, or other suitableembodiments for performing the described method. For example, a controldevice separate from the vehicle may be installed or otherwise coupledto the vehicle. Such control device may also be coupled to the one ormore LIDAR devices, coupled to a display (for rendering or displayingother information), and/or coupled to one or more other input/outputcomponents (such as a speaker, keypad, etc.). The vehicle 602 performingone or more steps may mean in some implementations that the controldevice performs one or more steps.

Beginning at 702, the vehicle 602 may receive data associated with amodulated optical signal emitted by a transmitter of a first LIDARdevice. For example, a device may be coupled to the LIDAR device 612,and a modulated optical signal may be transmitted from the LIDAR device614 or the LIDAR device 616 to the LIDAR device 612. The modulatedoptical signal includes an optical carrier signal modulated to include adata signal. The LIDAR device 612 or the device coupled to the LIDARdevice 612 may extract the data signal and determine the data includedin the data signal.

The vehicle 602 may also generate a rendering of the vehicle'senvironment based on information from one or more LIDAR devices coupledto the vehicle (704). For example, the vehicle 602 may include aprocessing system to receive measurements from the LIDAR device 612 whenthe LIDAR device 612 is in a detection mode. The measurements may beused to generate a rendering, such as a point cloud, a depth map, orother suitable representations of the environment 600. For example, theprocessing system may generate the example rendering 650 for theenvironment 600 (or the example rendering 690 for the environment 670)based on received measurements from the LIDAR device 612 or other LIDARdevices coupled to the vehicle, and the rendering may includerepresentations of surfaces detected in the environment 600 (orenvironment 670).

Based on the received data, the vehicle 602 may update the rendering(706). For example, the vehicle 602 may update a rendering of an objectin the environment (708). In some example implementations, the vehicle602 may receive an object identifier in the received data. For example,the LIDAR device 612 in a communication mode may receive, from the LIDARdevice 614 via a modulated optical signal, a packet including anidentifier for the vehicle 608. The identifier may be a specific vehicleID in the packet, a vehicle identification number, a make and model ofthe vehicle, or another suitable identifier.

The vehicle 602 may determine rendering information for the object basedon the identifier. Example rendering information may include dimensionsof one or more shapes in rendering the object (such as one or morerectangles or other shapes in rendering a vehicle), a representativeimage of the object (such as a representative drawing of a vehicle basedon, e.g., the type of vehicle, make and model of the vehicle, etc.), astock image of the object (such as a stock picture of a vehicle), or atexture to be applied to a rendering of the object (such as a texture tobe applied to the rendering of a vehicle). Rendering information mayalso indicate whether an object rendering is to be highlighted oradjusted (such as a change in size, dimensions, stretched, etc.). Insome implementations, if the vehicle 602 receives an identifier of thevehicle 608 (such as a make and model), the vehicle 602 may scan adatabase indexed by the identifier (such as makes and models) associatedwith rendering information (such as reference images of the vehicles,stored dimensions of the vehicles, highlighting information if anemergency vehicle, etc.) to determine the rendering information for thevehicle 608 (such as a stored image of the make and model of the vehicle608). The vehicle 602 may then update the object rendering based on thedetermined rendering information. For example, the rendered surfaces 658in the rendering 650 may be combined with a stored image or otherrendering information for the vehicle 608 to update the rendering 650.In another example, the rendered surface may be highlighted. In anotherexample of updating an object rendering, an object rendering may beadded to the rendering of the environment. For example, a rendering ofthe vehicle 676 and/or a rendering of the emergency vehicle 678 may beadded to the rendering 690.

In some other implementations, a LIDAR device may be configured totransmit a data signal including a specific description of how an objectshould be rendered. For example, the LIDAR device 614 of the vehicle 608may transmit the rendering model, the texturing, and any other featuresfor rendering the vehicle. In this manner, a memory including a look-uptable, database, etc. may not be used in determining the renderinginformation for the object in updating the rendering of the environment.

Referring back to FIG. 7, after updating the rendering, the vehicle 602may provide the updated rendering for display (710). For example, acontrol device coupled to the vehicle 602 may provide the updatedrendering to a display coupled to the vehicle 602, and the display maydisplay the updated rendering to a driver and/or passenger. In anotherexample, the rendering may be provided to a remote display for others tobe apprised of the environment of the vehicle 602.

FIG. 8 illustrates an example rendering 800 of the environment 600 inFIG. 6. The rendering 800 may be an example of an adjusted rendering ofrendering 650 in FIG. 6B based on rendering information for the vehicles608 and 610. For example, the vehicle 602 may store representativeimages (such as a stock image, line drawing, etc.) or other suitablerendering information for the received identifiers for the vehicles 608and 610. The LIDAR devices 614 and 616 may transmit vehicle identifiersto the LIDAR device 612, and the vehicle 602 may use the receivedvehicle identifiers to determine the rendering information for thevehicles 608 and 610 to adjust the rendering 650.

The rendered surfaces 658 associated with the vehicle 608 may becombined with the determined rendering information for the vehicle 608(such as a stock image or stored drawing for the vehicle 608). Forexample, an image of the vehicle 608 may be aligned with the renderedsurfaces 658 (such as by sizing and/or orienting the image of thevehicle 608 to align with the rendered surfaces 658). In the examplerendering 800, object 808 corresponds to the adjusted rendering of thevehicle 608. Similarly, object 810 in the rendering 800 corresponds tothe adjusted rendering of the vehicle 610.

While not shown, other objects that may be represented in the rendering650 may include infrastructure, buildings, road markings, pedestrians,or other non-vehicles whose rendering may be adjusted. In some examples,a LIDAR device coupled to a vehicle may transmit information regardingan object in the environment other than the vehicle. For example, one ormore LIDAR devices of the vehicle 608 (such as the LIDAR device 614) maybe used to determine depths of objects in the environment of the vehicle608, and one or more LIDAR devices of the vehicle 610 (such as the LIDARdevice 616) may be used to determine depths of objects in theenvironment of the vehicle 610. In one example, each vehicle 602, 608,and 610 may generate a rendering based on the measurements provided bythe LIDAR devices of the respective vehicles. In some implementations,the vehicles 602, 608, and 610 may share the LIDAR measurements with oneanother (via LIDAR communications), and the combined LIDAR measurementsmay be used by the vehicle 602 for generating a more comprehensiverendering of the environment 600 than rendering 650.

In some implementations, any suitable updated rendering may be displayedon a display of the vehicle 602. For example, an updated rendering maybe generated for an area of the environment not yet within range or LoSof the LIDAR devices for the vehicle 602. The vehicle 610 may detectobjects near the street behind the vehicle 610 (e.g., using the LIDARdevice 616), which may be used for rendering the environment 600 notwithin the field of view of the vehicle 612. The vehicle 610 maytransmit the rendering information for at least a portion of thatenvironment (such as rendering information for pedestrians, bicyclists,parked cars, traffic cones, obstructions, obstacles, etc.) to thevehicle 602. In this manner, the vehicle 602 may use the renderinginformation to include additional information about the environment 600on the right side of the rendering 650 (which is blocked by the building606 from the field of view of the LIDAR device 612 for the vehicle 602).In some aspects, rendering information may be shared among a pluralityof vehicles and infrastructure, and the rendering for a specific vehicle(such as the vehicle 602) may include areas well beyond the vehicle. Forexample, rendering information about the environment of a vehiclemultiple hops from the vehicle 610 may be transmitted through the hopsto the vehicle 610 and then to the vehicle 602. The renderinginformation may then be used by the vehicle 602 to expand the areacovered by the rendering generated by the vehicle 602.

In addition to updating a rendering, received data may be used fornavigation or other automated operations of an autonomous vehicle. Forexample, the updated renderings may indicate additional obstructions orobstacles, and the vehicle may thus navigate (such as update itsnavigation) to avoid the additional obstacles during operation.Referring back to the renderings 650 and 800 in FIGS. 6B and 8,respectively, the rendering 650 does not illustrate the rear portion ofthe vehicle 610, but the updated rendering 800 illustrates all of thevehicle 610. If the route of vehicle 602 would be determined to collidewith the end of vehicle 610 determined based on the received data, thenavigation of the vehicle 602 may be updated to avoid vehicle 610. Inanother example, a driver and/or passenger may be notified of a possiblecollision or obstruction based on the updated rendering. For example,the updated rendering including the obstructions or obstacles may bedisplayed to the driver and/or passengers, or the driver and/or thepassengers otherwise may be notified of the obstructions or obstacles.In this manner, the occupants of the vehicle may be apprised of changesin a route, the slowing of the vehicle, or other changes in navigationof an autonomous vehicle before the obstructions or obstacles are inview of the occupants. Additionally, or alternatively, a dispatcher orcentral office for a fleet including the vehicle may be notified of theobstructions or obstacles to explain the reasons for adjusting thenavigation of the vehicle.

In addition to updating the rendering 650 to include a model or otherrepresentative image of a vehicle (or another object in theenvironment), the rendering 650 may be updated to include a texture forthe model. For example, renderings of vehicles within the same fleet maybe adjusted to have a similar texture to indicate belonging to the samefleet.

FIG. 9A illustrates an example rendering 900 of the environment 600 inFIG. 6A where the vehicle 608 and the vehicle 602 belong to the samefleet. As shown, objects 902 and 908 in the rendering 900 may betextured (such as shaded) similarly. Object 910, corresponding to thevehicle 610 that is not part of the same fleet, may be textureddifferently than objects 902 and 908. For example, object 910 mayinclude a different shade than objects 902 and 908. Objects, such asinfrastructure, buildings, roads, vehicles, etc. may each be textureddifferently according to the type of object. For example, stands may berendered using the same texture, emergency vehicles may be renderedusing the same texture, etc. In this manner, a driver and/or passengermay easily identify similar objects in the environment based on thetextures in the displayed rendering. In some implementations, updating arendering may include highlighting an object rendering. For example,renderings of emergency vehicles may be highlighted to notify theoccupants of the vehicle of the presence of an emergency vehicle.Similarly, construction zones, safety zones, or other areas of theenvironment may be highlighted to notify the vehicle occupants of theirpresence in the environment.

FIG. 9B illustrates an example updated rendering 950 of the environment670 (FIG. 6C). The updated rendering 950 includes a rendering 952 of theemergency vehicle 678. The rendering 952 may be highlighted to indicatethat the vehicle 678 is an emergency vehicle. The rendering 952 may alsoinclude a representative image 954 included for the vehicle 674 and arepresentative image 956 included for the vehicle 676 not originallyincluded in the rendering 670.

Referring back to the environment 670 in FIG. 6C, vehicle 676 andemergency vehicle 678 may be occluded from the LoS of LIDAR devicescoupled to vehicle 672. As a result, the rendering 690 in FIG. 6D (basedon measurements from the LIDAR devices) does not include renderings ofthe vehicle 676 and the emergency vehicle 678. In some implementations,a LIDAR device coupled to the vehicle 672 may receive reflections of amodulated optical signal from a LIDAR device coupled to the vehicle 676or the emergency vehicle 678. The modulated optical signal may includedata indicating the presence of the emergency vehicle 678. The data mayalso indicate other information regarding the emergency vehicle, such asthe type of emergency vehicle, whether the emergency vehicle 678 isresponding to an emergency, and/or the path of travel, speed, etc. ofthe emergency vehicle. The data may also include information about othervehicles or objects. For example, the data may indicate a presence ofthe vehicle 676.

In some other implementations, the vehicle 672 may receive dataregarding the emergency vehicle 678 from an intermediate device. Thevehicle 676 may sense the presence of the emergency vehicle 678 (such asvia a LIDAR device coupled to the vehicle 676). The vehicle 676 maytransmit data regarding the emergency vehicle 678 to the vehicle 674within LoS via a modulated optical signal 680. The vehicle 674 may thentransmit the data regarding the emergency vehicle 678 to the vehicle 672within LoS via a modulated optical signal 682. While one intermediatenode for transmitting the data is shown, any number of hops may beperformed by the data. When the vehicle 672 receives the data regardingthe emergency vehicle 678 (and the data regarding the vehicle 676), thevehicle 672 may update the rendering 690 to fill-in portions of theocclusions caused by the building 684 as well as highlight or otherwiseindicate the presence of emergency vehicle 678. In addition oralternative to highlighting an emergency vehicle, the vehicle 672 mayotherwise notify a driver and/or passenger by e.g., flashing thedisplay, providing an audible notification, applying a special textureto the rendering, and/or providing a textual notification on thedisplay. While the examples in FIGS. 6C, 6D, and 9B are described withrespect to an emergency vehicle (such as an ambulance), the operationsmay apply to any vehicle, object, or area of interest (such asconstruction zones, school zones, accident areas, etc.).

In some implementations, rendering information for objects in anenvironment may be provided via a software as a service (SaaS) model.For example, a unique identifier for an object may be received by thevehicle 602. The vehicle 602 may then communicate with the service (suchas via a cellular modem or other communication means) to obtain therendering information for the object. For example, the vehicle 602 mayreceive a vehicle identification number of the vehicle 608, and thevehicle 602 may communicate with a remote server storing renderinginformation for a plurality of objects, including the vehicle 608. Theremote server may provide the rendering information for the vehicle 608in response to the request, and the rendering information may be used bythe vehicle 602 to update the rendering.

In some other implementations, the vehicle 602 may store renderinginformation for objects, and a remote server may provide renderinginformation of objects not locally stored for the vehicle 602. Inaddition, or to the alternative, a remote server may be used forupdating the locally stored rendering information. For example, when anew make and model of a vehicle is released, rendering information forthe vehicle may be created but not yet stored in a memory of the vehicle602. LIDAR communication may be used to update the locally storedrendering information of vehicle 602 to include the new renderinginformation. In some implementations, the vehicle 602 may update itsstored rendering information when coupled to a stand with a dedicatedbackhaul (such as the stand 408 in FIG. 4).

Other use cases for LIDAR communications exist, and the presentdisclosure is not limited to the above examples. For example, a device(such as a vehicle) may include software or firmware that periodicallyrequires updates. A LIDAR device may be used to receive software updatesfrom other devices. For example, a software update to include renderinginformation for a new make and model of a vehicle may be provided to afirst subset of vehicles. Those vehicles may then transmit the update toother vehicles in the environment via LIDAR communications. In thismanner, an update may be distributed quickly without requiring allvehicles to connect to a central location (such as a server storing theupdate). For example, communications between devices (such as vehiclesand infrastructure) may be similar to a peer to peer network, and theupdate may be propagated across devices without requiring a centralrepository for downloading the update. Another example use case mayinclude audio or video transmission between devices (such as for voicecalls, messaging, or videoconferencing).

In general, one or more operations may be based on data received via amodulated optical signal for LIDAR communications. As noted herein,LIDAR communications may be used to provide data for, e.g., affectingnavigation of an autonomous vehicle, providing notifications to a driveror passenger, adjusting access to locations, etc.

FIG. 10 shows a flow chart depicting an example operation 1000 forperforming one or more operations based on data received in atransmission from a LIDAR transmitter. While FIG. 10 is discussed interms of a device performing one or more operations for the purposes ofexplaining aspects of the disclosure. The operation 1000 may beperformed by a vehicle, infrastructure, a dispatcher or central officecoordinating a fleet of vehicles, a processing system (such as anentertainment system) coupled to a vehicle, a suitable control system(such as a control system coupled to the vehicle for Level 4 autonomousdriving), etc. The example operation 1000 is not limited to beingperformed by a specific device or for performing a specific deviceoperation.

Beginning at 1002, a device may receive data that is associated with amodulated optical signal emitted by a transmitting LIDAR device. Forexample, the device (such as a vehicle, processing system, etc.) may becoupled to a LIDAR device configured to receive a modulated opticalsignal from a separate transmitting LIDAR device. The receiver of thereceiving LIDAR device may receive the modulated optical signal, whichmay include an optical carrier signal modulated to carry a data signalincluding the data. The carrier signal may be the signal typically usedfor detection and ranging. The receiving LIDAR device (or the devicecoupled to the receiving LIDAR device) may demodulate the modulatedoptical signal to generate and provide the data provided in themodulated optical signal. In this manner, the device (such as aprocessing system or vehicle) may receive the data associated with themodulated optical signal. In some implementations, the data may includeone or more packets. Referring back to FIG. 3, example packets mayinclude information specific to, e.g., locating a vehicle, accessing aspecific area or location with a vehicle, object rendering, etc.

Referring back to FIG. 10, the device may determine, based on thereceived data, one or more operations to be performed (1004). Forexample, the device may determine one or more operations based on one ormore packets received. In some implementations, the device may determineone or more vehicle locator operations (1006). In one example, if avehicle receives information in the data that the vehicle is to belocated, the vehicle may determine to return to a designated spot (suchas a parking space including a stand, as illustrated in FIG. 4) andcommunicate with the stand (via LIDAR communications) the vehicle'slocation and current status. In another example, if the device receivesinformation about a missing vehicle and the device recently communicatedwith the missing vehicle, the device may communicate with a vehicle,infrastructure, a dispatch, etc., the last received location and otherinformation about the missing vehicle. Additionally, or alternatively,the device may propagate the missing vehicle message to other vehiclesand devices via LIDAR communications.

In some other implementations, the device may determine one or morevehicle hailing operations (1008). Referring back to FIG. 4, if thestand 408 is a walk-up stand for a fleet of automated taxis, the stand408 may communicate with a vehicle via LIDAR communications that thevehicle is assigned to a specific client and ride. The vehicle mayreceive the data from the stand 408 via LIDAR communications between aLIDAR device coupled to the vehicle and a LIDAR device coupled to thestand and located in a separate lot where the vehicle is queued. Basedon the received data, the vehicle may approach a space by the stand 408and communicate, with the stand 408, information regarding the ride orclient assigned to the vehicle. The stand 408 may notify the client thatthe vehicle has arrived so that the client may embark. In anotherexample, the vehicle or device may determine that a driver (such as ataxi driver) is to be notified of a client and/or ride assigned to thevehicle. The taxi driver may approach the client based on thenotification in order to begin the ride.

Referring back to FIG. 10, in some further implementations, the devicemay determine one or more notification operations (1010). In someexamples, referring back to FIG. 5, the vehicle 506 may receive datathat an ambulance 502 or other emergency vehicle is in the vicinity. Thevehicle 506 may pull to the curb, stop, or otherwise adjust thevehicle's operation or navigation to allow the ambulance 502 to safelypass. Additionally, or alternatively, the vehicle 506 or another devicemay determine to notify a driver and/or passengers of the presence ofthe ambulance 502. In some other examples, the vehicle 506 or device maydetermine to indicate the presence of the ambulance 502 to othervehicles, infrastructure, or other devices via LIDAR communications(thus propagating the indication to other vehicles and infrastructurewithin an area of the ambulance 502).

In another example of notification operations, a vehicle may receivedata regarding available parking spaces in a parking garage. The vehiclemay therefore drive to and park in one of the available spaces. Thevehicle or a device may determine to notify a driver and/or passenger ofthe available spaces (such as indicating available spaces on a displayedmap of the parking garage), etc. Other example notification operationsmay include updating a route based on a notification of an accident ortraffic along the current route, notifying other vehicles of theaccident or traffic via LIDAR communications, notifying a driver and/orpassenger of changes in the route or traffic conditions, etc.

Referring back to FIG. 10, in some other implementations, the device maydetermine one or more limited access area operations (1012). Forexample, if a city center is limited to autonomous vehicles, zeroemission vehicles, or other specific types of vehicles, a vehicle mayreceive data associated with the access restriction from a LIDAR devicecoupled to the vehicle. The LIDAR device may receive the data in amodulated optical signal transmitted by LIDAR device coupled toinfrastructure or another suitable device. For example, a gate, tollbooth, stand, etc. may include a LIDAR device to transmit an indicationof the access restrictions to the vehicle. The vehicle may determinewhether the vehicle is allowed in the area based on the indication. Thevehicle may also determine one or more navigation operations to accessthe area if the vehicle is allowed. For example, an autonomous vehiclemay enter a city center if access is limited to autonomous vehicles.Additionally, or alternatively, the vehicle may determine to notify adriver and/or passenger of an approaching limited access area and thelimitations for access. In some examples, the driver may determinewhether to access the area based on the notification. In some furtherexamples, the vehicle may be notified of a toll or charge for accessinga specific area. For example, a bridge, lane, or area of the city may beassociated with an access charge. The vehicle may provide account orother payment information via LIDAR communications before access toinfrastructure for the limited area (such as for providing payment to atoll booth before crossing a bridge). In some examples, the driverand/or passenger may be notified of the charge.

Since limitations for an area may change based on present circumstances(such as a time of day, day of the week, current traffic congestion,smog levels, a circumstance requiring limitation of an area to emergencyvehicles, etc.), such changes in limitations may be communicated. Inresponse, a vehicle may, e.g., determine to notify the driver and/orpassenger of changes in access limitations for the area, to adjust aroute to navigate around a newly restricted area, or to notify othervehicles of the changes via LIDAR communications.

Referring back to FIG. 10, in some further implementations, the devicemay determine one or more visualization or rendering operations (1014).For example, an environment of a vehicle, LIDAR device, infrastructure,or other suitable device may be rendered (such as generating a pointcloud, mapping of the environment, depth map, etc.), and the renderingmay be displayed to a driver and/or passenger. Referring back to FIGS.6A and 6B, the vehicle 602 (or another suitable device, such as anentertainment system, processing system, etc. coupled to the vehicle602) may generate the rendering 650 from data provided by the LIDARdevice 612. The vehicle 602 (or other suitable device) may also receiveone or more identifiers of the vehicles 608 and 610 in transmissionsfrom the respective LIDAR devices 614 and 616. The vehicle 602 (or othersuitable device) may therefore determine rendering information for thevehicles 608 and 610 based on the received identifiers (such as lookingup the information in a memory of the vehicle or device, communicatingwith a remote service to provide rendering information for one or moreobjects, determining rendering information encoded in the transmissionfrom the respective LIDAR device, etc.). Rendering operations maytherefore include using the determined rendering information to updatethe rendering 650. For example, rendering 650 in FIG. 6B may be updatedto rendering 800 in FIG. 8 based on representative images (e.g., models)for the vehicles 608 and 610 used to update the rendering. Other examplerendering operations may include applying a defined texture foremergency vehicles, applying similar textures for similar types ofobjects, applying a unique texture specific to each vehicle or object inthe environment and associated with a unique identifier (such as aunique Transmitter ID 302 in FIG. 3), etc. In some examples, the vehicle602 or other suitable device may determine that the updated rendering ofthe environment is to be displayed to a driver and/or passenger.Rendering operations may also include contacting a remote service toupdate a stored database of rendering information, updating the databasebased on receiving new rendering information (such as for a new make andmodel of a vehicle), or propagating updates to the rendering informationvia LIDAR communications with other vehicles or devices in theenvironment.

Referring back to FIG. 10, the device may then instruct the one or moreoperations be performed (1016). For example, if one or more renderingoperations are determined, the device may instruct a display or anotherdevice coupled to the display to display the rendering and display anupdated rendering. In another example, if the device is a vehicle andthe one or more operations are limited access area operations, thevehicle may instruct one or more vehicle components to navigate thevehicle into or out of the area. The device may also provideinstructions for displaying information regarding the limited accessarea to a driver and/or passenger.

While some of the example processes in the example operation 1000 inFIG. 10 are described from the perspective of a vehicle, any suitabledevice may perform one or more associated operations. For example, astand (or processing system coupled to the stand) may perform vehiclelocator operations (such as contacting a dispatcher) in response toreceiving a LIDAR transmission from a missing vehicle. In anotherexample, a stand may perform vehicle hailing operations (such asnotifying a client that the vehicle has arrived) in response toreceiving a LIDAR transmission from the hailed vehicle. In anotherexample, a stand may perform notification operations (such astransmitting a notification of an ambulance in the area) in response toreceiving a notification in a LIDAR transmission. In a further example,a toll booth or gate may perform limited access area operations (such aschanging an access fee or access restrictions for the area) in responseto receiving an update in traffic congestion from LIDAR devices forinfrastructure or vehicles in the area. As described, the exampleoperation 1000 is not limited to being performed by a specific device(such as a vehicle).

Additionally, while the example operations (such as ride hailing, accessrestriction, updated rendering, etc.) are described separately, a devicemay be configured to perform any number and combination of theoperations. For example, a device may be configured to update arendering regarding an emergency vehicle, notify the vehicle occupantsof the presence of an emergency vehicle, update an area restricted tothe vehicle based on the presence of the emergency vehicle, and updatenavigation of the vehicle to avoid the restricted area.

Those of skill in the art will appreciate that information and signalsmay be represented using any of a variety of different technologies andtechniques. For example, data, instructions, commands, information,signals, bits, symbols, and chips that may be referenced throughout theabove description may be represented by voltages, currents,electromagnetic waves, magnetic fields or particles, optical fields orparticles, or any combination thereof. Additionally, those of skill inthe art will appreciate that the examples are provided for illustrativepurposes in explaining aspects of the present disclosure. Aspects of thepresent disclosure may be performed by any suitable device (such as anautonomous vehicle, a fleet of autonomous vehicles, infrastructure, acontrol system for a vehicle or infrastructure, an entertainment systemcoupled to a vehicle, or other devices), and are not limited to specificexamples herein.

Further, those of skill in the art will appreciate that the variousillustrative logical blocks, modules, circuits, and algorithm stepsdescribed in connection with the aspects disclosed herein may beimplemented as electronic hardware, computer software, or combinationsof both. To clearly illustrate this interchangeability of hardware andsoftware, various components, blocks, modules, circuits, and steps havebeen described above generally in terms of their functionality. Whethersuch functionality is implemented as hardware or software depends uponthe particular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the disclosure. For example, a processing system of a vehicleor other suitable device may include one or more processors and a memorycoupled to the one or more processors. The memory may includeinstructions executed by the one or more processors to cause the deviceto perform operations as described herein. The processing system mayalso be coupled to one or more LIDAR devices for LIDAR communication anddetection and ranging of surfaces. In some aspects, the processingsystem may include dedicated hardware, such as one or more integratedcircuits, configured to perform one or more operations.

As such. the methods, sequences or algorithms described in connectionwith the aspects disclosed herein may be embodied directly in hardware,in a software module executed by a processor, or in a combination of thetwo. A software module may reside in RAM memory, flash memory, ROMmemory, EPROM memory, EEPROM memory, registers, hard disk, a removabledisk, a CD-ROM, or any other form of storage medium known in the art. Anexample storage medium is coupled to one or more processors (e.g.,directly connected or remotely coupled) such that the one or moreprocessors can read information from, and write information to, thestorage medium. In the alternative, the storage medium may be integralto the one or more processors.

In the foregoing specification, the example embodiments have beendescribed with reference to specific example embodiments thereof. Itwill, however, be evident that various modifications and changes may bemade thereto without departing from the broader scope of the disclosureas set forth in the appended claims. For example, while vehicles areillustrated in general as automobiles, any suitable vehicle may be used,such as a motorcycle, drone, aircraft, watercraft, helicopter, etc.Further, a vehicle may include one or more LIDAR devices, or the vehiclemay be coupled to a LIDAR device manufactured separately from thevehicle. The vehicle may also include one or more processing systems,entertainment systems, control systems, etc., or the vehicle may becoupled to such processing system, entertainment system, control systemmanufactured separately from the vehicle. For example, one or more LIDARdevices may be attached to different portions of an automobile to ensurecoverage of the environment surrounding the automobile. The LIDARdevices may be coupled to a processing system of the automobile (or acontrol system coupled to the vehicle) and provide data transmitted byother LIDAR devices for processing, or transmit data provided by theprocessing system to other LIDAR devices via a modulated optical signal.

The specification and drawings are, accordingly, to be regarded in anillustrative sense rather than a restrictive sense.

What is claimed is:
 1. A device comprising: one or more processors; anda memory coupled to the one or more processors, the memory includinginstructions that, when executed by the one or more processors, causethe one or more processors to: receive a modulated optical signalemitted by a light emitting transmitter of a first light detection andranging (LIDAR) device and received by a light detecting receiver of asecond LIDAR device coupled to a vehicle, wherein the modulated opticalsignal includes an optical carrier signal modulated to include a datasignal; demodulate the modulated optical signal to extract the datasignal, wherein the data signal includes data indicative of an object inan environment; generate a rendering of the environment of the vehiclebased on LIDAR information from one or more LIDAR devices coupled to thevehicle, wherein the LIDAR information comprises a plurality of datapoints associated with the environment configured to form a point cloud;update the rendering based on the data from the data signal, whereinupdating the rendering includes modifying the rendering to include anobject rendering of the object in the environment; provide the updatedrendering for display on a display coupled to the vehicle; receive asecond modulated optical signal received by the second LIDAR device;demodulate the second modulated optical signal to extract a second datasignal; and update one or more entries in a database based on theextracted second data signal.
 2. The device of claim 1, wherein updatingthe rendering includes at least one from the group consisting of:highlighting the object rendering during display; adjusting texture ofthe object rendering; including a representative image of the object inthe rendering of the environment; or adjusting the dimensions of theobject rendering.
 3. The device of claim 2, wherein displaying theupdated rendering includes notifying at least one of a driver or apassenger in the vehicle that the object in the environment is anemergency vehicle, wherein the modulated optical signal is demodulatedto extract the data signal, wherein the data signal comprises a datapacket, and wherein the data packet is decoded based on a communicationprotocol.
 4. The device of claim 3, wherein execution of theinstructions further causes the one or more processors to: determine oneor more navigation operations of the vehicle to be adjusted based on theemergency vehicle in the environment; and provide the one or moreadjusted navigation operations for execution by the vehicle.
 5. Thedevice of claim 1, further comprising a second receiver configure toreceive a wireless signal, wherein the data signal includes data thatoriginates from a second device not coupled to the first LIDAR device,and wherein the data signal comprises a point-to-point communication. 6.The device of claim 2, wherein execution of the instructions furthercauses the one or more processors to: search for the indication of theobject in a database indexed by vehicle identifiers, wherein theindication of the object includes a vehicle identifier; and identify, inthe database, an indication of the update to be performed to the objectrendering associated with the vehicle identifier, wherein updating theobject rendering is based on the identified indication of the update tobe performed.
 7. The device of claim 1, wherein: the one or more LIDARdevices coupled to the vehicle includes the second LIDAR deviceoperating in a detection mode to detect reflected signals from theenvironment; and the second LIDAR device operates in a communicationmode when receiving the modulated optical signal from the first LIDARdevice.
 8. A non-transitory computer readable medium storinginstructions that, when executed by one or more processors, cause theone or more processors to: receive a modulated optical signal emitted bya light emitting transmitter of a first light detection and ranging(LIDAR) device and received by a light detecting receiver of a secondLIDAR device coupled to a vehicle, wherein the modulated optical signalincludes an optical carrier signal modulated to include a data signal;demodulate the modulated optical signal to extract the data signal,wherein the data signal includes data indicative of an object in anenvironment; generate a rendering of the environment of the vehiclebased on LIDAR information from one or more LIDAR devices coupled to thevehicle, wherein the LIDAR information comprises a plurality of datapoints associated with the environment configured to form a point cloud,update the rendering based on the data from the data signal, wherein theupdating the rendering includes modifying the rendering to include anobject rendering of the object in the environment; provide the updatedrendering for display on a display coupled to the vehicle; receive asecond modulated optical signal received by the second LIDAR device;demodulate the second modulated optical signal to extract a second datasignal; and update one or more entries in the database based on theextracted second data signal.
 9. The non-transitory computer readablemedium of claim 8, wherein updating the rendering includes at least onefrom the group consisting of: highlighting the object rendering duringdisplay; adjusting texture of the object rendering; including arepresentative image of the object in the rendering of the environment;or adjusting the dimensions of the object rendering.
 10. Thenon-transitory computer readable medium of claim 9, wherein displayingthe updated rendering includes notifying at least one of a driver or apassenger in the vehicle that the object in the environment is anemergency vehicle.
 11. The non-transitory computer readable medium ofclaim 10, wherein execution of the instructions further causes the oneor more processors to: determine one or more navigation operations ofthe vehicle to be adjusted based on the emergency vehicle in theenvironment; and provide the one or more adjusted navigation operationsfor execution by the vehicle.
 12. The non-transitory computer readablemedium of claim 9, wherein the data signal comprises at least one datapacket that originates from a second device not coupled to the firstLIDAR device.
 13. The non-transitory computer readable medium of claim9, wherein execution of the instructions further causes the one or moreprocessors to: search for the indication of the object in a databaseindexed by vehicle identifiers, wherein the indication of the objectincludes a vehicle identifier; and identify, in the database, anindication of the update to be performed to the object renderingassociated with the vehicle identifier, wherein updating the objectrendering is based on the identified indication of the update to beperformed.
 14. A method comprising: receiving, by one or moreprocessors, a modulated optical signal emitted by a light emittingtransmitter of a first light detection and ranging (LIDAR) device andreceived by a light detecting receiver of a second LIDAR device coupledto a vehicle, wherein the modulated optical signal includes an opticalcarrier signal modulated to include a data signal; demodulating themodulated optical signal to extract the data signal, wherein the datasignal includes data indicative of an object in an environment;generating, by the one or more processors, a rendering of theenvironment of the vehicle based on LIDAR information from one or moreLIDAR devices coupled to the vehicle, wherein the LIDAR informationcomprises a plurality of data points associated with the environmentconfigured to form a point cloud; updating the rendering based on thedata from the data signal, wherein updating the rendering includesmodifying the rendering to include an object rendering of the object inthe environment; providing the updated rendering for display on adisplay coupled to the vehicle; receiving a second modulated opticalsignal received by the second LIDAR device; demodulating the secondmodulated optical signal to extract a second data signal; and updatingone or more entries in the database based on the extracted second datasignal.
 15. The method of claim 14, wherein updating the renderingincludes at least one from the group consisting of: highlighting theobject rendering during display; adjusting texture of the objectrendering; including a representative image of the object in therendering of the environment; and adjusting the dimensions of the objectrendering.
 16. The method of claim 15, wherein displaying the updatedrendering includes notifying at least one of a driver or a passenger inthe vehicle that the object in the environment is an emergency vehicle.17. The method of claim 15, wherein the data signal comprises at leastone data packet that originates from a second device not coupled to thefirst LIDAR device.
 18. The method of claim 15, further comprising:searching for the indication of the object in a database indexed byvehicle identifiers, wherein the indication of the object includes avehicle identifier; identifying, in the database, the indication of theupdate to be performed to the object rendering associated with thevehicle identifier, wherein updating the object rendering is based onthe identified indication of the update to be performed.